Difficulty of Interruptions 1 Does the Difficulty of an Interruption Affect our Ability to Resume? David M. Cades George Mason University Deborah A. Boehm Davis George Mason University J. Gregory Trafton Naval Research Laboratory Christopher A. Mo nk George Mason University Research has shown that different types of interruptions can affect their disruptiveness. However, it is unclear how different features of the interrupting task determine its disruptive effects. Specifically, some theories predict that the difficulty of an interruption does not contribute to the disruptive effects of that interruption alone. Disruptive effects can be mediated by the extent to which the interrupting task interferes with the ability to rehearse during the interruption. In this experiment participants performed a single primary task with three interruptions of different difficulty. We found that interruptions were more disruptive when the task minimized the participant’s ability to rehearse (as measured by the number of mental operators required to perform the task) and not just when they were more difficult. These results suggest that the ability to rehearse during an interruption is critical in facilitating resumption of a primary task. INTRODUCTION As we are faced with more and more sources of information vying for our attention at any given time, it is becoming increasingly important to understand how interruptions affect our abilities to complete tasks. Not surprisingly, many studies of the effects of interruptions have shown them to be disruptive to the performance of a primary task (Gillie & Broadbent, 1989; Miyata & Nonnan, 1986; Monk, 2004; Trafton, Altmann, Brock, & Mintz, 2003). However, little is known about the role played by the various features of interrupting tasks such as modality, similarity, or difficulty. This paper seeks to examine the difficulty aspect in order to better understand how this aspect makes interruptions more or less disruptive. Although no comprehensive theory of interrupted task perfonnance currently exists, the goal-activation model (Altmann & Trafton, 2002) does make predictions of how disruptive an interruption will be. Simply, the model suggests that the disruption will be greater the longer an interruption is and the less a person rehearses the primary task during the interruption. The model, however, does not make any specific predictions related to how other features of an interrupting task such as similarity and difficulty, will affect disruptiveness beyond how those aspects interact with interruption duration and rehearsal. Other research on interruptions has suggested that disruptiveness is directly related to the difficulty of the interruption, regardless of whether a person has the opportunity to rehearse (Gillie & Broadbent, 1989). It is important to note that as the mental complexity, or amount of mental effort, of the secondary task increases, the opportunity to rehearse the primary task decreases. This paper examines the extent to which difficulty of the interrupting task disrupts primary task performance, with a particular focus on the role that opportunity for rehearsal play in detennining the disruptiveness of the interruption. EXPERIMENT In order to examine how the difficulty of an interruption, presumably through its interference with the opportunity to rehearse, affects the resumption of a task, we conducted an experiment in which participants perfonned a primary task with interruptions of three levels of difficulty. In this Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 2007 2. REPORT TYPE 3. DATES COVERED 00-00-2007 to 00-00-2007 4. TITLE AND SUBTITLE Does the Difficulty of an Interruption Affect our Ability to Resume? 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Naval Research Laboratory,Navy Center for Applied Research in Artificial Intelligence (NCARAI),4555 Overlook Avenue SW,Washington,DC,20375 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) 11. SPONSOR/MONITOR'S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES Human Factors and Ergonomics Society 51st Annual Meeting, Oct 1-5, 2007, Baltimore, MD 14. ABSTRACT 15. SUBJECT TERMS 1 1 16. SECURITY CLASSIFICATION OF: a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified 17. LIMITATION OF ABSTRACT Same as Report (SAR) 18. NUMBER OF PAGES 5 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 Difficulty of Interruptions 2 experiment, all participants performed three sessions of a primary task with interruptions. In one condition they were interrupted with a simple shadowing task, in which they repeated numbers aloud read to them by the computer. In the other two interruption conditions, they were interrupted with variations of the n-back working memory task (Lovett, Daily, & Reder, 2000). Both n-back tasks required participants to listen to a series of numbers read aloud by the computer, to make judgments as to whether or not the most recently read number was higher or lower than one of the previously read numbers and, finally, to acknowledge their choice by clicking on either a “Higher” or “Lower” button located at the top of the screen. In the easy (1-back) task, they were asked to compare the most recent number to the one just before it and in the difficult (3-back task), they were asked to compare the most recent number to the one three numbers prior. COGNITIVE TASK ANALYSIS On the surface, it would appear that the 3-back task was more difficult than the 1-back task, which in turn was more difficult than the shadowing task. This surface appearance is based on the fact that it should be easier to remember and compare two number read consecutively (1-back) than to compare a two numbers separated by two other numbers (3-back), and that both of these tasks should be easier than simply repeating a number (shadowing). However, in order to understand whether each of these tasks might allow for rehearsal it is important to look at the resources that each requires. An NGOMSL (Natural Language Goals, Operators, Methods, Selection Rules) (Kieras & Poison, 1985) task analysis was performed on the three interruption tasks (see Table 1) to determine the likelihood that participants could rehearse while performing that task. Both the 1-back and 3-back tasks were found to have three mental operators, while the shadowing task had zero. The presence of the mental operators in the 1-back and 3-back tasks would suggest that participants needed to maintain information in memory, which would likely reduce rehearsal ability during these interruptions. The lack of any mental operators in the analysis of the shadowing task suggests that participants did have at least the opportunity to rehearse during the interruption. Task/Actions Operator 1-Back Table 1: NGOMSL analysis of Shadowing, 1-Back, and 3-Back interrupting tasks Listen to 1st Number Perceive Listen to 2nd Number Perceive Remember 1st Number Mental Compare 2 Numbers Mental Decide if 2nd Number is Higher or Mental Lower Move mouse to proper button Point Click Button Click 3-Back Listen to 1st Number Perceive Listen to 2nd Number Perceive Listen to 3rd Number Perceive Listen to 4th Number Perceive Remember 1st Number Mental Compare 1st and 4th Numbers Mental Decide if 4th Number is Higher or Lower Mental Move mouse to proper button Point Click Button Click Shadowinq Listen to 1st Number Perceive Say 1st Number Thus, if the opportunity to rehearse has a direct impact on people’s ability to resume a task following an interruption, as is predicted by the memory for goals model (Altmann & Trafton, 2002), the NGOMSL analysis suggests that the 1- back and 3-back tasks would show more disruptive effects than the shadowing task. In other words, people would resume fastest in the shadowing condition and slower in the two n-back conditions. Alternatively, if difficulty of the interrupting task has a direct role in detennining the interruption’s disruptiveness, we expect that the shadowing task would be the least disruptive, followed by the 1- back task, and then the 3-back task. Specifically, participants would resume fastest in the shadowing condition, slower in the 1-back condition, and slowest in the 3-back condition. METHOD Difficulty of Interruptions 3 Participants Thirty-six undergraduates from George Mason University participated in this experiment for class credit. All were randomly assigned to either the shadowing, the easy //-back, or the difficult //-back interruption condition. Task and Materials The primary task (see Figure 1) consisted of programming a Video Cassette Recorder (VCR) interface to record a specific television program in the future. The interruption tasks consisted of a simple number shadowing task (the easiest condition) and two variations of the //-back working memory task (Lovett et ah, 2000), a 1-back task (the medium difficulty condition) and a 3-back task (the hardest condition). The VCR interface was programmed in Macintosh Common Lisp, was designed for experimental use, and was not based on any specific VCR model (Gray, 2000). In order to program a show on the VCR, participants were given a 3x5 index card with the name, start time, end time, day of the week, and channel of a television program. The programming task was completed once all of this information was entered into the computer. During all interruption tasks, numbers ranging from one to nine were read aloud by the computer at a rate of one number every three seconds. Each interruption lasted for thirty seconds. Participants were interrupted an average of eleven times per session. Design and Procedure The experiment was a 3 x 3 mixed factorial design with interruption difficulty as a between subjects factor with three levels (Shadowing, 1- back, and 3-back) and sessions as a within subjects factor with three levels (1,2, and 3). Participants were trained on the VCR task individually and then the VCR task with either the shadowing, 1-back, or 3-back interruption depending on what condition they were in. Participants then completed three sessions, with each session consisting of three different television shows to program. Each session lasted approximately fifteen minutes and contained Figure 1: The VCR Interface an average of eleven 30 second interruptions. Interruptions were triggered by a random number of mouse clicks on the VCR task ranging from fifteen to twenty two; however this was not apparent to the participants. At the onset of the interruption, the VCR interface would disappear and a new screen would be presented with the “Higher” and “Lower” buttons on the top. After thirty seconds this screen would disappear and the VCR interface would return. A short break was given between each session. Measures Each mouse action was time-stamped and recorded for all participants. The inter-action interval represents the amount of time between any two actions on the primary VCR task. A resumption lag is a special type of inter-action interval taken by measuring the action time between the ending of the interrupting task and the first action back on the primary task. This measure (Altmann & Trafton, 2002) has been used accurately to quantify the disruptive effects of interruptions in the past (Monk, 2004; Trafton et ah, 2003). RESULTS AND DISCUSSION Resumption lags below 200 milliseconds were removed from the data because they likely were anticipatory clicks resulting from the fact that the //- back interrupting tasks required participants to click very close to the time when the interruption ended and the VCR task reappeared on the screen. Following this, outliers greater than three standard Difficulty of Interruptions 4 deviations from the mean were removed, which constituted 1.1% of the total data. The following results suggest that the difficulty alone does not dictate how disruptive an interruption is. If difficulty alone causes disruption, then we would expect the most difficult (3-back) condition to show the slowest resumption lags, the easy /7-back (1-back) intermediate resumption lags, and the easiest (shadowing) condition, the fastest resumption lags. A repeated measures ANOVA, with interruption difficulty as a between-subjects factor, did show a significant main effect for interruption difficulty (F( 2, 33) = 7.83, p < .01, MSE= 1,062,687, rf = .32). However, although participants resumed significantly faster in the Easy (shadowing) condition than in either the easy //-back (1-back) (p < .01) or the hard /7-back (3-back) conditions ip < .01), resumption times in the hard /7-back (3-back) and easy /7-back (1-back) conditions were not statistically different (p = 1.0) based on Tukey HSD post hoc comparisons (see Figure 2). Thus, these data suggest that while people are disrupted by interruptions in all three of these conditions (resumption lags are longer than inter-action intervals), interruption difficulty can not be the sole reason for the disruptiveness. ■ Hard n-back (3-back) □ Easy n-back (1-back) Figure 2: Average Resumption Lag by Session by interruption Difficulty (Error Bars are Standard Error) If difficulty of the task only plays a role to the extent that the task prevents rehearsal, as suggested by Altmann & Trafton (2002), we might expect a different outcome. The NGOMSL analysis suggested the 1-back task was sufficiently taxing to minimize rehearsal of the primary task during the interruption. If this is true, the 3-back task (which will also minimize rehearsal) should not lead to decreased resumption ability over and above the 1- back task according to the memory for goals model. The analysis also suggested that the shadowing task, with no mental operators, would be more likely to allow participants to rehearse. Thus, this theory would predict that people should be able to resume the primary task better following an interruption with the shadowing task. When compared to both the 1-back and 3-back conditions, the resumption lags in the shadowing condition indeed were significantly faster across all three sessions, supporting the interpretation that difficulty may affect disruptiveness through the prevention of rehearsal. Additionally, the mean inter-action intervals decreased linearly across sessions for all conditions (F(l, 34) = 17.95,// < .001, MSE = 87,772, rf = .35). This confirmed the practice effect for the primary task shown repeatedly in interrupted task performance and suggested that any differences in resumption perfonnance was not due to an interaction between the interruption type and the primary task (Cades, Trafton, & Boehm-Davis, 2006; Trafton et al., 2003). These data also show that, as in Cades et al. (2006), participants resumed faster with more practice on the interruption (see Figure 2), suggesting further that interruption type does not affect people’s ability to improve over time at dealing with interruptions. When collapsed across all conditions, resumption lags decreased linearly across sessions (F( \, 33) = 13.59,// < .001, MSE = 230,530, rf = .29). GENERAL DISCUSSION Although we cannot detennine from our data whether participants were rehearsing during any of the interruption conditions, we can be sure that there is more to assessing the disruptiveness of an interruption then just examining its difficulty alone. Our data showed that simply repeating numbers is less disruptive than having to make any type of comparison, whether it is one that places a relatively low load on working memory or a slightly higher one. Difficulty of Interruptions 5 Both the 1-back and 3-back tasks required three mental operators. It may be that had we used a fourth task that required additional mental operators, we would have disrupted resumption ability beyond the levels shown here. However, it could be that once rehearsal has been minimized it really does not matter how much additional burden the interrupting task places on the participant. The implication of this work is that we cannot simply say that more difficult interruptions will lead to greater disruptions. Rather, we must consider other features of the interruption to gain a full understanding of how disruptive a particular task will be. REFERENCES Altmann, E. M., & Trafton, J. G. (2004). Task interruption: Resumption lab and the role of cues. Paper presented at the Proceedings of the 26th annual conference of the Cognitive Science Society. Cades, D. M., Trafton, J. G., & Boehm-Davis, D. A. (2006). Mitigating disruptions: Can resuming an interrupted task be trained? Paper presented at the Proceedings of the Human Factors and Ergonomics Society 50 th Annual Meeting. Gillie, T., & Broadbent, D. (1989). What makes interruptions disruptive? A study of length, similarity, and complexity. Psychological Research, 50, 243-250. Gray, W. D., (2000). The nature and processing of errors in interactive behavior. Cognitive Science, 24(2), 205- 248. Kieras, D. E., & Poison, P. G. (1985). An approach to the formal analysis of user complexity. International Journal of Man-Machine Studies, 22, 365-394. Lovett, M. C., Daily, L. Z., & Reder, L. M. (2000). A source of activation theory of working memory: Cross-task prediction of performance in ACT-R. Journal of Cognitive Systems Research, 1, 99-118. Miyata, Y., & Norman, D. A. (1986). Psychological issues in support of multiple activities. In D. A. N. a. S. W. Draper (Ed.), User centered system design (pp. 265- 284). Hillsdale, NJ: Lawrence Erlbaum Associates Inc. Monk, C. A. (2004). The effect of frequent versus infrequent interruptions on primary task resumption. Paper presented at the Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting. Trafton, J. G., Altmann, E. M., Brock, D. P., & Mintz, F. E. (2003). Preparing to resume an interrupted task: Effects of prospective goal encoding and retrospective rehearsal. International Journal of Human Computer Studies, 55(5), 583-603.